Skip to content

test: skip flaky hotplug tests on 5.10 hosts#6007

Merged
ShadowCurse merged 2 commits into
firecracker-microvm:mainfrom
not4s:fix/skip-flaky-virtio-mem-hotplug
Jun 30, 2026
Merged

test: skip flaky hotplug tests on 5.10 hosts#6007
ShadowCurse merged 2 commits into
firecracker-microvm:mainfrom
not4s:fix/skip-flaky-virtio-mem-hotplug

Conversation

@not4s

@not4s not4s commented Jun 30, 2026

Copy link
Copy Markdown
Contributor

Changes

  • Skip test_virtio_mem_hotplug_hotunplug and test_memory_hotplug_latency on 5.10 hosts.
  • Add RUSTSEC-2026-0190 to .cargo/audit.toml ignore list.

Reason

Both tests intermittently fail on 5.10 hosts, where the GET /hotplug/memory API call exceeds the maximum duration assertion (the unplug request gets queued behind the VMM thread and the API call waits). Skip until the RC is investigated.

The cargo-audit ignore is bundled here to unblock the merge queue: RUSTSEC-2026-0190 (anyhow) was published and is failing test_cargo_audit on all PRs. anyhow is not a direct dependency of any Firecracker crate; it is only pulled in transitively by build-time WASM tooling (wit-bindgen).

License Acceptance

By submitting this pull request, I confirm that my contribution is made under
the terms of the Apache 2.0 license. For more information on following Developer
Certificate of Origin and signing off your commits, please check
CONTRIBUTING.md.

PR Checklist

  • I have read and understand CONTRIBUTING.md.
  • I have run tools/devtool checkbuild --all to verify that the PR passes
    build checks on all supported architectures.
  • I have run tools/devtool checkstyle to verify that the PR passes the
    automated style checks.
  • I have described what is done in these changes, why they are needed, and
    how they are solving the problem in a clear and encompassing way.
  • I have updated any relevant documentation (both in code and in the docs)
    in the PR.
  • I have mentioned all user-facing changes in CHANGELOG.md.
  • If a specific issue led to this PR, this PR closes the issue.
  • When making API changes, I have followed the
    Runbook for Firecracker API changes.
  • I have tested all new and changed functionalities in unit tests and/or
    integration tests.
  • I have linked an issue to every new TODO.

  • This functionality cannot be added in rust-vmm.

@codecov

codecov Bot commented Jun 30, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 83.08%. Comparing base (7503f25) to head (f450115).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #6007   +/-   ##
=======================================
  Coverage   83.08%   83.08%           
=======================================
  Files         277      277           
  Lines       30201    30201           
=======================================
  Hits        25094    25094           
  Misses       5107     5107           
Flag Coverage Δ
5.10-m5n.metal 83.41% <ø> (-0.02%) ⬇️
5.10-m6a.metal 82.77% <ø> (ø)
5.10-m6g.metal 80.05% <ø> (-0.01%) ⬇️
5.10-m6i.metal 83.41% <ø> (-0.01%) ⬇️
5.10-m7a.metal-48xl 82.75% <ø> (-0.01%) ⬇️
5.10-m7g.metal 80.06% <ø> (ø)
5.10-m7i.metal-24xl 83.38% <ø> (-0.01%) ⬇️
5.10-m7i.metal-48xl 83.38% <ø> (-0.02%) ⬇️
5.10-m8g.metal-24xl 80.05% <ø> (-0.01%) ⬇️
5.10-m8g.metal-48xl 80.05% <ø> (-0.01%) ⬇️
5.10-m8i.metal-48xl 83.38% <ø> (-0.01%) ⬇️
5.10-m8i.metal-96xl 83.39% <ø> (ø)
6.1-m5n.metal 83.44% <ø> (-0.01%) ⬇️
6.1-m6a.metal 82.79% <ø> (-0.01%) ⬇️
6.1-m6g.metal 80.05% <ø> (-0.01%) ⬇️
6.1-m6i.metal 83.44% <ø> (+<0.01%) ⬆️
6.1-m7a.metal-48xl 82.78% <ø> (ø)
6.1-m7g.metal 80.05% <ø> (-0.01%) ⬇️
6.1-m7i.metal-24xl 83.44% <ø> (-0.01%) ⬇️
6.1-m7i.metal-48xl 83.45% <ø> (-0.01%) ⬇️
6.1-m8g.metal-24xl 80.05% <ø> (-0.01%) ⬇️
6.1-m8g.metal-48xl 80.05% <ø> (-0.01%) ⬇️
6.1-m8i.metal-48xl 83.45% <ø> (ø)
6.1-m8i.metal-96xl 83.45% <ø> (ø)
6.18-m5n.metal 83.43% <ø> (-0.02%) ⬇️
6.18-m6a.metal 82.79% <ø> (ø)
6.18-m6g.metal 80.05% <ø> (-0.01%) ⬇️
6.18-m6i.metal 83.43% <ø> (-0.01%) ⬇️
6.18-m7a.metal-48xl 82.78% <ø> (-0.01%) ⬇️
6.18-m7g.metal 80.05% <ø> (-0.01%) ⬇️
6.18-m7i.metal-24xl 83.44% <ø> (-0.01%) ⬇️
6.18-m7i.metal-48xl 83.45% <ø> (-0.01%) ⬇️
6.18-m8g.metal-24xl 80.05% <ø> (-0.01%) ⬇️
6.18-m8g.metal-48xl 80.05% <ø> (-0.01%) ⬇️
6.18-m8i.metal-48xl 83.46% <ø> (+<0.01%) ⬆️
6.18-m8i.metal-96xl 83.45% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@not4s not4s marked this pull request as ready for review June 30, 2026 10:22
@not4s not4s added the Status: Awaiting review Indicates that a pull request is ready to be reviewed label Jun 30, 2026

@zulinx86 zulinx86 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought we only want to skip it on 5.10 host. Does it also fail on 6.1/6.18 hosts?

@not4s

not4s commented Jun 30, 2026

Copy link
Copy Markdown
Contributor Author

@zulinx86 Nope just the 5.10. Should we just skip it for 5.10 hosts?

Comment thread tests/integration_tests/performance/test_hotplug_memory.py Outdated
@not4s not4s force-pushed the fix/skip-flaky-virtio-mem-hotplug branch from a29587b to 7531d6e Compare June 30, 2026 10:36
@not4s not4s changed the title test: skip flaky test_virtio_mem_hotplug_hotunplug test: skip flaky hotplug tests on 5.10 hosts Jun 30, 2026
@not4s not4s requested review from Manciukic and zulinx86 June 30, 2026 11:06
zulinx86
zulinx86 previously approved these changes Jun 30, 2026
test_virtio_mem_hotplug_hotunplug and test_memory_hotplug_latency
intermittently fail on x86_64 5.10 hosts, where the GET /hotplug/memory
API call exceeds the maximum duration assertion. Skip both there until
the latency is investigated.

Signed-off-by: Jay Chung <jaehoc@amazon.com>
@not4s not4s force-pushed the fix/skip-flaky-virtio-mem-hotplug branch from 7531d6e to a574423 Compare June 30, 2026 11:32
ShadowCurse
ShadowCurse previously approved these changes Jun 30, 2026
zulinx86
zulinx86 previously approved these changes Jun 30, 2026
@not4s not4s enabled auto-merge June 30, 2026 11:38
@not4s not4s added this pull request to the merge queue Jun 30, 2026
@github-merge-queue github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Jun 30, 2026
@not4s not4s added this pull request to the merge queue Jun 30, 2026
@github-merge-queue github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Jun 30, 2026
@not4s not4s added this pull request to the merge queue Jun 30, 2026
cargo-audit flags RUSTSEC-2026-0190, failing the security audit on all
branches. anyhow is not a direct dependency of any Firecracker crate;
it is only pulled in transitively by build-time WASM tooling
(wit-bindgen), so we are not affected. Ignore it to unblock CI.

Signed-off-by: Jay Chung <jaehoc@amazon.com>
@not4s not4s dismissed stale reviews from zulinx86 and ShadowCurse via f450115 June 30, 2026 13:33
@not4s not4s removed this pull request from the merge queue due to a manual request Jun 30, 2026
@ShadowCurse ShadowCurse enabled auto-merge June 30, 2026 14:00
@ShadowCurse ShadowCurse added this pull request to the merge queue Jun 30, 2026
Merged via the queue into firecracker-microvm:main with commit 573d249 Jun 30, 2026
9 checks passed
@not4s not4s deleted the fix/skip-flaky-virtio-mem-hotplug branch June 30, 2026 14:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Status: Awaiting review Indicates that a pull request is ready to be reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants